The increasing number of scientific publications has made bibliographic search engines essential tools in all disciplines. These software-based devices, however, are far from perfect. Comparisons of software-based bibliographic search engines with complete lists of three authors' publications showed that reference citations were not generally available before 1970, and that the effectiveness of recovery was improving but was quite variable, yielding on average 36 percent of the publications. There was marked year-to-year inconsistency in the recovery of titles. The inconsistency could not be explained by differences in indexing due to journal reputation: there was no evident relationship between search effectiveness and journal impact factor, but the percentage of recovered citations was higher for indexed journals. Search engines are widely used in bibliographic searches performed for evaluating researchers, awarding promotions, or assessing journal performance. Given the ineffectiveness of search engines, their use in making such important personal and institutional decisions needs careful consideration.